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A Dynamic Program Assessment Framework for Learning Communities 


Abstract 

This research builds upon Malnarich, Pettitt, and Minos (2014) investigation of students’ reflections on their 
learning community (LC) experiences. Adapting their Peer-to-Peer Reflection Protocol for use at 
Kingsborough Community College, CUNY, we present a framework for dynamic LC program assessment. To 
obtain feedback about theory-practice connections in our English as a Second Language (ESL) LCs, students 
were asked to consider whether their experiences reflected the sociocultural tenets underlying the program 
(Vygotsky, 1978, 1986). Students provided individual responses in writing and engaged in a communal 
conversation facilitated by a professor, program director, and campus administrator based on these writings. 
Using conversation analysis, our examination of responses to one program principle in the discussion context 
uncovered two related phenomena: students talked about what collaboration meant to them in their LC 
(collaboration-in-content), and also engaged in the act of collaboration in the conversation itself 
(collaboration-in-process). The exploratory nature of these interactions as part of a growing community 
support and extend the findings of Malnarich et al. (2014). We conclude by arguing for the continued 
investigation of ways to bridge theory and practice in our LC program work by bringing assessment activity 
and classroom activity together, and inviting various campus stakeholders to this dynamic process. 
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Introduction 


Inspiration for the Study 

Collaborative learning activities through which “learning unfolds in the 
most public of ways” (Smith & MacGregor, 1992, p. 10) are at the heart of the 
learning community (LC) movement. In the LC mission, collaborative learning 
activity is foundational to a curriculum reimagined as collectively constructed and 
dynamically shared. However, Malnarich, Pettitt, and Mino (2014) point to a 
fundamental mismatch between this view and typical assessment practices in 
learning community settings. “We were struck by an anomaly,” they write. 

“While LCs are associated with collaborative learning, LC assessment of student 
learning, in keeping with assessment practices in general, elicits individual not 
collective responses” (p. 22). Malnarich et al.’s (2014) study involved the use of 
an online student survey addressing students’ experiences in learning 
communities from both two- and four-year institutions. In addition to the 
quantitative data collection, the investigation included students’ qualitative 
written responses about their LC participation at four campuses, with students 
further engaging in reflective discussions based on the written feedback they had 
provided. Findings from a case study at one of these schools, Holyoke 
Community College, were provocative: students’ group discussions were rich, 
extending the content of their individual responses and mirroring the kinds of 
educational discussions they reported having in their LC classrooms as they 
probed their understandings of individual knowledge building as a shared activity. 
As the authors report: “exploratory talk, or the facilitated discussion itself, 
promoted the collective construction of knowledge, thus refining, deepening, and 
transforming students’ understanding of their learning experience in LCs” (p. 18). 

Based upon their study results, Malnarich, Pettit and Mino (2014) put forth a 
Peer-to-Peer Reflection Protocol (PRP) and encouraged other learning community 
practitioners to draw upon it in the assessment of their own LC programs. By 
implementing the PRP and refining its prompts over time, the authors note, “we 
hope that, as a field, we will discover more about students’ individual and 
collective experiences of collaborative learning” (p. 22). The research that follows 
takes up this call. Building upon Malnarich, Pettit and Mino’s investigation, and 
adapting their PRP for our program, this study explores the development and 
implementation of a framework for dynamic program assessment in English as a 
Second Language (ESL) Learning Communities at Kingsborough Community 
College, CUNY. At the heart of our analysis are two central questions concerning 
theory and practice: how are we doing, as administrators and teachers, at enacting 
the sociocultural theoretical principles grounding our learning community 
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program design? And, based on students’ exploratory reports on their experiences, 
should we be rethinking any aspects of these program principles and/or practices? 

ESL Learning Communities at Kingsborough: A Sociocultural Approach 

Kingsborough Community College’s commitment to learning communities 
dates back to the 1990s with the inception of the Intensive ESL Program. This 
highly successful semester-long offering served as a springboard for additional 
LC programs at the college for a variety of populations, and established 
Kingsborough’s national reputation in the field (Babbitt, 2006; Babbitt & 
Mlynarczyk, 2000; Song, 2006). Beginning in 2010, however, program 
administrators began to re-evaluate aspects of the ESL LC structure and 
curriculum, focusing on both qualitative and quantitative data collection. Based 
on reports from English faculty, ourselves included, gaps seemed to be widening 
between the standards of our departmental assessment protocols, including 
reading and writing exams and an essay portfolio, and students’ demonstration of 
the ability to meet these standards. Speech professors were also reporting that 
they did not have enough time to work with students on both accuracy and 
fluency of expression. At the same time, data from Kingsborough’s Office of 
Institutional Research showed a significant number of students exiting from the 
ESL Learning Community environment with good grades but not thriving in the 
same way in subsequent developmental English coursework. Upon graduating 
from the highest level of the three-tier ESL sequence and taking classes with 
native English speakers, too many students were receiving Repeat grades, often 
multiple times. In a number of these cases, students dropped out of the college, 
seemingly because they had lost their financial aid. 

To explore and work to address these complex issues, we formed a small 
interdisciplinary group of ESL Learning Community faculty and administrators, 
representing Departments of English, Communications and Performing Arts, and 
Behavioral Sciences. Despite our challenges, we knew that the linked program 
model was more effective for our community college students than stand-alone 
coursework; our aim was not to abandon the one-semester curriculum, but rather 
to enhance it to better meet our learners’ needs. We turned to our experiences as 
educators and also to scholarship, sharing our own classroom narratives and 
revisiting the sociocultural tenets underlying the learning community movement 
and current best practices in second language education. What emerged from 
these conversations was a blueprint for a two-semester, inter-leveled program 
model based on our renewed commitment to two foundational beliefs: 1) thinking, 
learning, and the use of language are interwoven processes that cannot be viewed 
in isolation from one another, and 2) new knowledge is socially constructed 
between experts and novices over time (Vygotsky, 1978, 1986). 


http://washingtoncenter.evergreen.edu/lcrpjournal/vol4/iss2/2 
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As we considered modifications to our program structure and curriculum, 
we went back to the central ideas of the psychologist Lev Vygotsky, whose work 
is often cited as foundational to the learning community reform movement. 
Drawing upon Marxian principles in post-revolutionary Russia, Vygotsky’s 
research explored the social origin of the human mind. His central insight was that 
human mental processes are mediated by our engagement with culturally 
produced sign systems, including oral and written language. It is through our 
relationships with other people and cultural artifacts—our “lived experiences” 
(Moll, 2014)—that Vygotsky believed our intellectual capacities take shape. In 
this view, a learner’s environment is not just a setting for development, but rather 
its very foundation. 

Vygotsky’s genetic law of cultural development states: “every function in 
the child’s cultural development appears twice: first, on the social level, and later, 
on the individual level; first, between people ( interpsychologiccil ), and then inside 
the child {intrapsychological)" (Vygotsky, 1978, p. 57). Internalization is 
therefore understood to be gradual process whereby activity on the external plane 
becomes transformed as new knowledge inside a learner. This process does not 
occur in a linear or measured fashion but rather is believed to evolve 
unpredictably and dynamically— “the result of a long series of developmental 
events ” (Vygotsky, 1978, p. 57). From a sociocultural perspective, humans’ use of 
language not only allows individuals to construct meaning jointly, it also serves as 
a tool for thought as communication becomes internalized, regulating individuals’ 
cognition “through the discourse of inner speech” (Moll, 2014, p. 33). This 
internal dialogue becomes transformed once again as it is externalized through 
dialogue in the social sphere. Our voices, thoughts, and actions are seen as 
mutually constitutive of one another. 

Drawing upon Vygotsky’s ideas, we made two foundational shifts to our 
program structure. The first sought to honor the notion of the time required for 
learning to occur, and the inextricable links between reading, writing, speaking, 
and listening for literacy development. Maintaining the rigorous schedule of the 
fall semester— including courses in ESL, Speech, an academic discipline 
(Sociology, Psychology, History, or Health Education) a Freshman Seminar, an 
Integrative Language Seminar, and a tutoring component—a second program 
term was added. The ESL course would continue into the spring, linked with an 
additional Speech class and continued tutoring. 1 The second shift, reflective of 
our understanding of knowledge building as co-constructed by novices and 
experts, was inter-leveled student placement. Rather than designating students as 
members of an ESL “level” within a hierarchical sequence, we brought students 


1 The revised program structure also functions to support students’ broader integration into the 
college, as they are able to enroll in courses of their choosing outside of their learning community 
in Winter and Spring semesters. 
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together into a single learning community cohort. If students were found to 
require English language support based on their incoming CUNY Reading and 
Writing exam scores, they would bring their unique strengths and weaknesses 
together, encouraged to assist one another in collaboration with their professors 
and tutors, throughout the yearlong LC experience. Our commitment to a dialogic 
view of education had additional practical impacts on the program curriculum as 
well, including the transformation of the tutoring experience into a “Reading Lab” 
for students’ extensive reading, an emphasis on project-based activities, and the 
integration of multimodality through digital storytelling. 

The revised LC structure was piloted in two learning communities in 2012- 
2013 and now serves as the model for all ESL links in Kingsborough’s Learning 
Community Program. Before expanding the two-semester program, we measured 
whether or not the modifications we had made were successful. This assessment 
began with our broadest, most quantitatively measurable goal: to prepare students 
for successful entrance into credit-bearing English classes at the college. Students 
participating in the two-semester LC experience were found to accelerate through 
their English coursework at faster rates than students enrolled in one-semester 
ESL LCs, and demonstrated significant pre- and post-program gains in CUNY 
Reading and Writing exam scores. 2 We also have students fill out qualitative 
surveys, and reports at the end of the two-semester experience were positive. 
However, while such outcomes proved useful at showcasing the effectiveness of 
the program for outside evaluators—producing “creditable summary data” for 
college administrators, funders, and other LC stakeholders (Malnarich, Pettitt, & 
Mino, 2014, p. 9)—as administrators and teachers we found there to be an 
incongruity between the nature of these data and the work we had done to 
redesign our program based on theory. While the quantitative and qualitative 
results looked good on paper, they did not provide us with enough feedback on 
how well we were doing at enacting our sociocultural principles in the classroom, 
or whether or not, in fact, these were the principles we should be using as a guide 
to maximize students’ opportunities for learning. Program assessment that would 
result in meaningful curricular and pedagogical reflection required a more 
dynamic approach. 

Malnarich, Pettitt, and Mino’s (2014) Peer-to-Peer Reflection Protocol (see 
Appendix A) spoke to us as a tool we might use to engage in program assessment 
at this deeper level. The development of the PRP arose from a validation study of 
the Washington Center’s Online Survey of Students’ Experiences of Learning in 
Learning Communities. Lollowing the administration of this survey, researchers 
at participating colleges also had students produce responses in writing about 


2 In the first two years of two-semester ESL Learning Communities, the majority of students 
(51.7%) exceeded the pace they would have achieved had they progressed at expected levels in 
stand-alone classes. 
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select survey items, and engage in a follow up facilitated discussion with faculty 
members or designees who recorded proceedings through note taking. In addition 
to providing individual feedback, students were therefore given the opportunity to 
“develop a collective understanding of their learning community experience” 
(Malnarich et al., 2014, p. 20). The PRP suggests two stages for facilitating 
discussions. First, students are introduced to the whole process. Next, they engage 
in guided discussions with four main phases: 1) a listening round with students 
asked to share examples from their written responses; 2) a “making connections” 
phase, with students invited to point out relationships between ideas or common 
themes from the round robin sharing; 3) “wondering,” with students asked to raise 
questions or aspects of what they noticed relative to the learning experiences 
described; and 4) “uncovering complexity,” with participants invited to consider 
what “lies beneath the surface” of their experience. Finally, they are invited to 
appreciate each other for their contributions and their shared insights. 

Results from Malnarich, Pettitt, and Mino’s (2014) work showed students 
exploring not only their experiences in learning communities but also how these 
experiences reflected an underlying view of education as a social event. Our 
commitment to uncovering information from our students about the relationship 
between theory and practice in our LC program led us to the adaptation of this 
instrument for our own use. The Protocol also spoke to us as a tool we might use 
to blur the line that is typically drawn in higher education settings between 
assessment and teaching. A growing body of research in second language 
classrooms based on Vygotsky’s work considers teaching and assessment to be 
shared practices. For dynamic assessment to occur (Lantolf & Poehner, 2011; 
Poehner, 2008; Poehner & Lantolf, 2010), teachers are called upon, through 
dialogue, to at once evaluate students’ needs, orient to these needs, and work to 
guide students towards development. Inspired by this work, we were interested in 
exploring a methodology for program assessment that was educational as well as 
evaluative. Could assessment be instantiated as a cooperative dialogue that 
provides learning opportunities for students, teachers, and administrators to best 
meet students’ needs at a programmatic level? 

The research questions thus serving to set this study in motion were: 

• How might we use and adapt Malnarich, Pettitt, and Mino’s (2014) Peer- 
to-Peer Reflection Protocol to dynamically assess our LC program from 
multiple perspectives? 

• What might we learn about the quality of our program through our use 
of this feedback tool in order to improve it? 

• Might findings from communal assessment activity at our community 
college support Malnarich, Pettitt, and Mino’s? 
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• What additional insights might we uncover by examining participants’ 
communal conversations on a turn-by-tum basis? 

A Methodology for Dynamic Program Assessment 

Our methodology for a dynamic learning community program assessment, 
based on Malnarich, Pettitt, and Mino’s (2014) research included an individual 
writing and a reflective discussion component. In their English classes, students in 
two ESL Learning Community sections (n=16) were asked, near the end of their 
two-semester participation, to write individually, with evidence to support their 
ideas, about whether or not their experiences in the LC program reflected four 
learning principles of the program: 

1. Language learning takes a long time. 

2. Reading - a lot - is important. 

3. Students with different strengths and weaknesses should work together, 
in collaboration with teachers and tutors. 

4. When we learn a new language we construct a new identity. 3 

Using Malnarich et al.’s (2014) Peer-to-Peer Protocol as a guide, students 
were invited to participate in a communal conversation based on these writings 
during a later class session. Various LC stakeholders at the college were included 
in this activity: students, their English professors over the full year of the 
program, and administrators. One or both ESL Learning Communities co¬ 
directors and Kingsborough’s Associate Director of Institutional Research (IR) 
were present for the conversations. Both co-directors had also served as the 
students’ English teachers: one for the fall semester only, the other for the 
duration of the program. Our intention in bringing these individuals together was 
not only to encourage a meaningful conversation about the LC experience that 
would result in insights beyond what students could bring about individually, but 
also to situate students’ responses within the context of a dialogue enriched by a 
diverse community of learners with various points of view. As teachers and 
administrators we saw ourselves also as learners, with the aim of being educated 
by our students in order to improve our shared program through a dynamic 
program assessment. 

Communal conversations began with a program co-director introducing the 
IR professional to the group, and briefly explaining the activities of the day. 
Students’ reflective writings were then returned, and as a review, individual 
students were called upon to read each of the four program principles aloud. 
Students were next instructed to form pairs and discuss their written responses to 


3 We worked to ensure that the language we used to describe our program principles would be 
accessible to students. 
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one program principle with a partner. Participants then formed a circle and 
students, in a listening round, were asked to share with the whole group what had 
been reviewed with their peers. Students, in moving from pair work to sharing out 
with the class, engaged in a process that mirrored the discussion format that had 
become familiar to them in their learning community classrooms. We intended for 
students to experience this conversation not as a formal assessment exercise, 
removed from educational activity, but rather as another opportunity to build 
knowledge. 

The next phase of activity aimed to deepen our collective understandings of 
the relationship between theory and practice in our program. Students were first 
asked to connect common themes they had heard in the listening round. 

Professors and administrators subsequently followed up with questions and 
comments in an effort to point out new connections, or to solidify ones students 
had already made, based on the four program principles. While the IR 
professional, program co-director, and ESL instructor facilitated the conversation, 
their prompts and comments were intentionally non-prescriptive, allowing 
students to freely initiate questions, explore ideas of their choosing, and build 
upon others’ contributions. At the conclusion of the session, students were invited 
to suggest improvements for the program and thanked for their participation. 

The student participants were diverse. Many were from Central Asia and 
former Soviet Union nations, while others were from Bangladesh, Haiti and 
China. Their language abilities were at various stages of development. Some were 
full-time students of college age; others were parents who worked in addition to 
attending classes. Participating teachers and administrators also brought their 
unique sociocultural, positional, and professional identities and histories into the 
conversations. English instructors had a particularly close relationship with 
participating students having spent eight hours each week with them for a 
semester or year, and therefore brought with them an intimate knowledge of 
students’ personalities, idiosyncrasies, and talents. ESL LC co-directors, who 
understood the history of the program as well as its goals and challenges, viewed 
the conversations through this lens as well. While the IR professional was the 
only participant entirely unknown to the students prior to the conversations, as a 
foreign-born native-Spanish speaker he was the only facilitator who shared a 
similar language acquisition history with participating students. 

A critical component of our program assessment procedure was audio¬ 
recording the communal conversations. Both classroom discussions, 
approximately three hours total, were audio-recorded. We chose this method for 
data collection in order to capture and honor students’ contributions as part of the 
group—turn by speaking turn. Audio-recording also allowed us to capture all 
contributions made by the non-student participants. We could not predict how the 
discussions would unfold, or how conversational content might be influenced by 
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students’ relationships with one another or the presence of the research team. By 
audio-recording, we were able to ensure that noteworthy aspects of the 
conversations would not be lost. 

After we collected the data, we reviewed students’ written responses, 
looking for common themes, and we compiled rough transcripts and transcript 
summaries of the recorded conversations. Our purpose at this stage of analysis 
was to arrive at a general understanding of what was important to the participants. 
We then looked more closely for evidence of what Malnarich, Pettitt, and Mino 
(2014) noted as “the most significant finding to surface” from students’ reports in 
their study: “that collaborative learning activities and classroom conversations 
...promoted the collective construction of knowledge, that is, students making 
meaning together, thus deepening students’ understanding of the subject matter, 
themselves, and the world around them” (p. 19). To assess whether or not 
collaborative activity had a similar impact on the students in our study, we looked 
for places in both their individual writings and in the conversation transcripts 
where students were responding to our third program principle: Students with 
different strengths and weaknesses should work together, in collaboration with 
teachers and tutors. In the transcripts, we also looked for moments where students 
were visibly expanding their thinking about their learning community experiences 
as part of the group dialogue. We flagged notable issues and dynamic exchanges 
that arose from this initial review of the assessment materials. 

Highlighted moments became starting places for more detailed transcription, 
following conversation analysis conventions (see Table 1). Conversation analysis 
transcripts aim to represent real-world events as closely as possible, including 
notations for pauses, overlaps, and laughter, to show how participants negotiate 
their communication contingently through turn taking (Sacks, Schegloff, & 
Jefferson, 1974). 4 Researchers in second language classrooms have used 
conversation analysis to examine how teachers and students make particular 
aspects of language relevant to one another on a moment-by-moment basis (Kahn, 
2015; Ohta, 2001; Waring, 2016). By examining select data excerpts through this 
microanalytic lens, we were looking for insights into how students might co¬ 
construct their interpretation of the theory undergirding our learning community 
program as the result of taking part in a collaborative discussion. 

Findings and Analysis 

Collaboration-in-Content and Collaboration-in-Process 


4 In transcribing students' written language, we also chose not to gloss what the students had 
produced, representing this language exactly as it had been written. 
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One central finding from students’ reflective contributions in both 
formats—individually written and produced as part of a communal 
conversation—was that collaboration is important to them for learning. This 
phenomenon, collaboration-in-content, refers to students’ articulation of the 
significance of collaboration to them within their LC experience. However, while 
meaningful examples of collaboration-in-content were found throughout our 
dataset, the communal conversations were shown to elicit deeper and more 
nuanced explorations of the meaning of collaboration to students and the layered 
effects that working closely with others had on their education. This finding 
extends Malnarich, Pettitt, and Mino’s (2014) insight that facilitated discussions 
“turned out to be even more generative than writing a response to a prompt on 
one’s own” (pp. 19-20), as the discussions “encouraged further elaboration with 
examples and complicated students’ reported experiences” (p. 18). 

In their individual written reports, students had positive things to say in 
response to our stated program principle: Students with different strengths and 
weaknesses should work together, in collaboration with teachers and tutors. 
However their assessments in this context tended towards the general. Examples 
such as: “cooperative learning helps us leam more” and “we had a lot of fun 
doing assignments, doing group works, helping us to understand each other 
more,” highlighted the value of the kinds of active pedagogies emphasized in our 
program without providing much explanation of why or how. In particular, 
students noted the importance of assistance from other individuals—peers, tutors, 
and teachers—in their LC classrooms. One comment, “we had the same teachers 
and tutors. They knew me openly. They will better understand what I was trying 
to said,” appeared to address the value of relationship building over time with 
instructors and support staff in enhancing their quality of communication. This 
statement was followed by an example, “especially, Prof. Tara,” who “helped me 
a lot at writing, and understanding.” However, the student did not elaborate on the 
topic further. 

Unsurprisingly, as students were asked to complete this piece of writing as 
an assignment in English class, their individual responses also revealed an 
understanding of their writing activity as an academic task. Produced at the end of 
two semesters of English coursework, responses showcased students’ developing 
identities as academic writers aware of the conventions of essay composition. 
Examples can be seen in students’ use of collaboration-in-content language. One 
learner shared: “we get warms not only from each other, but also from our 
professors and tutors. They are always ready to help us with the challenges that 
appear in our path.” The student is expressing the positive feelings she has as a 
result of working with her learning community peers and mentors through the use 
of a correlative conjunction often seen in formal writing contexts to connect two 
related pieces of information (“we get warms not only from each other, but also 
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from our professors and tutors”). We also see her use of metaphoric language: 
professors and tutors are available to lend support with the challenges that appear 
in our path. Painting a picture of her and her classmates’ experience as a journey 
with roadblocks that assistors help them overcome, the student sophisticatedly 
employs a literary tool employed by expert writers to engage their audiences. 
However, her paragraph ends there. Examples such as these, while notable, gave 
the students’ writings a forced quality. Students seemed more interested in 
impressing their audience—understood to be members of an academic 
community—than in providing raw data about their LC experiences. 

In the communal conversation context, however, students’ collaboration-in¬ 
content voices were less polished and more exploratory, as the following excerpt 
demonstrates: 

Table 1 

Conversation analysis transcription symbols adapted from Atkinson & Heritage, 

1984, and Seedhouse, 2004 


overlapped speech 
= latch or continuing turn 

(4.0) interval between utterances, in seconds 
(.) very short untimed pause 

easy speaker emphasis 

my::: sound lengthening 

sudden cutoff 
? rising intonation 

, continuing intonation 

falling intonation 
< > slowed speech 

> < quickened speech 

( ) unclear speech 

(guess) transcriber’s uncertainty 
.hh in-breath 

hh out-breath 

(( )) nonverbal actions or transcriber’s comments 
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Figure 1 
Excerpt 1 


[Excerpt 1 50.13] 


01 

Student 1: 

1 think the biggest possibility from 

02 


this class is that we had the f- (.) 

03 


freedom of speech that we can ask 

04 


teachers if you don’t understand it. 

05 


They can share their- we can share 

06 


our thoughts, and they can they can 

07 


give us our (0.4) their thoughts and 

08 


how to make it better or change. 

09 


That’s all. Because (that’s true for) 

10 


every class we had. 

11 

Chris: 

urn hm 

12 

Gabrielle: 

hmm 

13 

Student 1: 

cuz 1 heard some friends who had like 

14 


the other Englishes, their teachers 

15 


are touqh they don’t like to suqqest 

16 


thouqht 


The student in Excerpt 1 provides an explanation of learning community 
collaboration and what it means to him. His point, shared in Lines 1-10, appears 
to be that the “freedom of speech” he experienced in his classes was the 
program’s most beneficial aspect, helping him and his classmates “understand” 
course content better (“we can ask teachers if you don’t understand it”), and 
allowing for more tailored feedback through open communication (“we can share 
our thoughts, and they can give us.. .their thoughts and how to make it better or 
change”). Conversation analysis details show that making this argument, the 
student exhibits moments of tentativeness in his pauses (Lines 2 and 7) and self¬ 
corrections (Lines 5 and 7). The contribution is less formal than the collaboration- 
in-content examples using academic language we saw in students’ written work, 
and appears to be unplanned. The emergent nature of this contribution can also be 
seen in the voices of two other participants in Lines 11 and 12: Chris, the 
Institutional Research administrator, and Gabrielle, the students’ first-semester 
English teacher and program’s co-director. Their motivational backchannel 
responses (“urn hm,” “hmm”) serve as prompts for the student to continue 
(Tannen, 2005), and in Lines 13-16, the student does just that, providing 
additional explanation about the nature of collaboration in his learning community 
by contrasting his experience with those of his friends in other English classes 
(“their teachers are tough they don’t like to suggest thought”). This elaborative 
statement was not present in his original contribution, but grew dynamically from 
the developing discussion. 
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Throughout the communal conversation data, we see such instances—of 
students not only talking about the importance of collaboration to them in 
meaningful ways, but also engaging in the act of collaboration on a turn-by-tum 
basis, a finding we are calling collaboration-in-process. The spontaneous 
contributions of teachers, administrators, and other students in the room are found 
to stretch students’ reflections on the learning community experience, and as this 
occurs, the formation of a new community that includes these varied participants 
can be seen in the unfolding discourse. In Excerpt 1, a student probes an aspect of 
his linked program participation—freedom of speech—as he explains what the 
term means to him and then develops his idea in response to others’ supportive 
utterances. These utterances, produced by a Kingsborough administrator and 
professor, contribute to the creation of a new community by demonstrating their 
alignment with the student and giving him an expert role in the conversation. It is 
the student who serves as “teacher” here, educating others about what is important 
to him as a college student and receiving the message that what he has to say is 
valued. The collaboration-in-process phenomenon continues in Excerpt 2 as a 
new student takes the floor in response to the feedback shared by Student 1. 


Figure 2 
Excerpt 2 


[Excerpt 2 51.36] 

01 Student 2: 


I’m agree with Student 1 that uh 
you are giving us freedom that we can 
express our selves , .hh like even like 
tho- even like with our digital 
movies we can like s ee::. (.) like 
try look at ourself from different 
sides (.) and you also look at us 
like from different size .hhh 
sides 
hhhhhh 
m [hm 


02 

03 

04 

05 

06 

07 

08 


09 Student 3: 

10 (Student): 

11 Gabrielle: 

12 Student 4: 

13 Student 2: 


[sides 

sides. A:nd urn (0.4) also when we’re 
uh write es says you can uh you: say 
you have to uh add your own 
experience, your opinion, but like 
ssome of professors ours are say 
nobody cares about your opinion you 
write with (who [you are) like give= 


14 

15 

16 

17 

18 
19 


20 Chris: 

21 Student 2 

22 Student 5 

23 Student 2 

24 


=But here we ex press , we have you 
know freedom. 


=like facts not your opinions 


=Yeah= 


[hm 
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Excerpt 2 shows a student building upon her peer’s contribution in Excerpt 
1 by affirming his freedom of speech idea in Lines 1-8 (“I’m agree with Student 1 
that uh you are giving us freedom that we can express ourselves”), and applying it 
to a new aspect of LC collaboration significant to her: the creation of their “digital 
movies.” The redesign of our ESL Learning Communities included a new project 
in the two-semester curriculum: digital storytelling. Our primary objective in 
implementing this assignment was to encourage students’ development of self in 
and through their second language by investing in the language, not only 
academically, but also emotionally, culturally, and socially (Kramsch, 2010). In 
this particular LC link, students produced and narrated a story in a multimodal 
format about a life-changing event in their lives and applied a concept from their 
Psychology class to the narrative. Student 2’s statement—that the freedom 
afforded by this project allowed her and her classmates to see themselves from 
varying perspectives (“we can like see like try look at ourself from different 
sides”), and also allowed professors to view them in new ways (“and you also 
look at us like from different size”)—provides evaluative feedback about a key 
curricular component of the program. The digital storytelling project promoted 
personal and relational investment as was our intention. We also see the 
unrehearsed, exploratory nature of this feedback in the student’s use of the filler 
“uh” in Line 1, her false starts in Lines 3-4 (“even like tho- even like with”) and 
Lines 5-6 (“we can like see like try look at ourself’), and her pauses and in¬ 
breaths. The gradual unfolding of this contribution, and the imprint of other 
voices upon it, can additionally be seen when a classmate, Student 3, points out 
that the word “size” should be pronounced “sides” in Line 9 and Student 2 
incorporates this correction into her developing thinking in Line 13. Another 
student, Student 4, is found to utter this corrected word in Line 12 (“sides”), 
indicating that it is notable to him as well. Students are demonstrating their co¬ 
construction of knowledge in action. 

Student 2 provides a second example in L ines 13-16 of how her learning 
community professors work with learners to encourage freedom of expression: in 
their essays (“also when we’re uh write essays.. .you say you have to uh add your 
own experience your opinion”). Like the student in Excerpt 1, she adds weight to 
the value of this kind of collaboration by contrasting it with the actions of other 
professors at the college—in this case professors she and her peers have taken 
classes from—who say that “nobody cares about your opinion” (Line 18), and 
who expect students to “give...facts not your opinion” (Lines 19 and 21). 5 After a 
new student chimes in with an agreement token in Line 22 (“yeah”), Student 2 
reiterates the value of creative activity in her learning community: “but here we 


5 Students in ESL Learning Communities have the opportunity to choose additional courses in 
Winter and Spring semesters which operate autonomously from the LC. 
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express, we have you know freedom.” 

In Excerpt 2, we see the instance of collaboration-in-content and 
collaboration-in-process initiated in Excerpt 1 develop and deepen. Students 
continue to address collaboration in the content of their contributions, and, as a 
new student builds upon a peer’s interpretation of what collaboration looks like in 
their classrooms, a fuller picture is painted of the original view. Freedom of 
speech in this learning community is a layered experience. It concerns the 
openness of expression between learners and teachers to clarify one another’s 
thinking and make improvements to students’ work, as Student 1 put forth. It also 
addresses the kinds of assignments given by linked instructors—digital stories and 
essays—that call upon students to share their experiences and opinions, as shared 
by Student 2. This contribution of Student 2, as demonstrated with Student 1 in 
the previous example, is not monologic, but dialogic. Interjections by other 
participants include corrective pronunciation feedback, uptake of this feedback 
not only by Student 2, but also another learner, and an agreement token. As in the 
previous excerpt, we also see supportive backchannel responses from Gabrielle 
and Chris (Lines 11 and 20). In the moment-by-moment development of one 
student’s commentary, we see evidence of collaboration-in-process as the 
individuals in this group do the very activity they are talking about: co¬ 
constructing meanings and sharing aspects of themselves. 

Thus far we have seen the presence of a learning community 
professor/program administrator and college administrator in the conversational 
background, supportively signaling to students that they are active listeners. As 
the topic of one particular English course text emerges in the excerpt that follows, 
the voices, and personalities, of non-student participants in the group become 
more visible. 


http://washingtoncenter.evergreen.edu/lcrpjournal/vol4/iss2/2 
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Figure 3 
Excerpt 3 


[Excerpt 3 34.39] 


01 

Student 6: 

now in this semester we had one book 

02 


that we didn’t like, (.) 

03 


Slaughterhouse [Five, 

04 

John: 

[heh heh heh heh (.) 

05 


heh heh 

06 

(Students): 

((laughter)) 

07 

Student 6: 

we ask professor= 

08 

Gabrielle: 

((referring to Chris)) =he likes [it 

09 

Chris: 

[one 

10 


of my favorite books 

11 

(Students): 

((escalating laughter)) 

12 

Student 6: 

oh my God 

13 

John: 

so::: infamous 

14 

Chris: 

that’s a hard book (.) that’s a 

15 


really tough book 

16 

John: 

but they liked a harder book >well 

17 


anyway sorry< 

18 

Group: 

((laughter, commenting)) 

19 

Student 3: 

we didn’t understand the humor of the 

20 


writer so 

21 

Chris: 

hm[mmm 

22 

Gabrielle: 

[mmmm 

23 

Chris: 

yeah it’s very subtle 

24 

(Students): 

((laughter)) 

25 

Student 6: 

so we stopped ((laughing voice)) 

26 


reading this book 

27 

Chris: 

wuhhow 

28 

Student 6: 

and uh (.) yeah professor Keller 

29 


started giving us short stories, (.) 

30 


and uh 1 think they also really 

31 


interesting, and it’s good that he 

32 


change this book and now we like the 

33 


stories that we read and we enjoy 

34 


reading it= 

35 

Student 2: 

=yehehs 


Excerpt 3 begins with Student 6 confessing a problem from the current 
semester of their linked program: students were asked to read Slaughterhouse 
Five, and this book was not well received (“now in this semester we had one book 
we didn’t like.. .Slaughterhouse Five”). The laughter that ensues in Lines 4-6, 
initiated by John, the English professor who introduced the book, and followed by 
multiple students, indicates that this shared experience was a source of humor for 
them. In Line 7, Student 6 makes an attempt to continue her story, but Gabrielle 
interrupts, pointing out that Chris, the IR administrator present for the 
conversation, actually likes this book (“he likes it”). Chris, in partial overlap with 
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Gabrielle admits this himself in Lines 9-10 (“one of my favorite books”), sparking 
escalating laughter from multiple students, and a disbelieving exclamation from 
Student 6, “oh my God,” in Line 12. John follows with what is seemingly a joking 
comment (“soooo infamous”), with Chris then found to express his understanding 
about why the book could be disliked (“that’s a hard book.. .that’s a really tough 
book”). In Lines 16-17 John appears to challenge Chris by stating that the 
students liked an even more difficult text than Slaughterhouse Five (“but they 
liked a harder book”). However, his dramatic tone—with word stress on “harder” 
and quickened pace in the seeming apology and attempt to change the subject that 
follows (“well anyway sorry”)—indicate that he may be speaking in jest, an 
interpretation confirmed by the participants when additional laughter and 
commentary follow his turn in Line 18. 

The levity in this episode continues. Unidentifiable students erupt in 
laughter again in Line 24 after Student 3, seemingly in an attempt to explain the 
group’s negative reaction to this book, shares in Lines 19-20 that it was author 
Vonnegut’s sense of humor that mystified them (“we didn’t understand the humor 
of the writer”), and Chris agrees in Line 23 that this humor is not easy to grasp 
(“yeah it’s very subtle”). Within this sequence, we also see Chris and Gabrielle 
demonstrate they are listening to Student 3 in their supportive backchannels. In 
Lines 25-26 Student 6 finally gets back to her story by revealing that 
Slaughterhouse Five was abandoned by the group (“so we stopped reading this 
book”), eliciting an elongated “wow” amazement token by Chris (“wuhow”)—an 
exclamation that can serve to develop a storyteller’s narrative by encouraging the 
teller to continue (Norrick, 2000). Student 6 does further explain the events in her 
story in Lines 28-34 that follow: there was a book change in the course 
(“Professor Keller started giving us short stories”), and there were positive effects 
resulting from the change: students are happy with the “really interesting” short 
stories they are reading now (“we enjoy reading it”)—a sentiment validated by 
another student, laughingly, in Line 35 (“yehehs”). 

In Excerpt 3, participants exhibit a lighter tone than we have seen in 
Excerpts 1 and 2. Their comedy, however, is reflective of the same interactional 
phenomena we have been seeing. Collaboration-in-content is showcased as 
students address a real curricular conflict and shed light on how the issue, with the 
support of their instructor, was resolved. A new aspect of meaningful 
collaboration within this learning community is uncovered: students’ ability to 
influence the very content of their course. While this episode could have 
undermined the LC experience for both students and teachers, it appears from 
their reactions in the telling that it had the opposite effect. Participants’ sense of 
community seems to have grown as students have taken ownership over their 
learning process with their professor’s assistance: their laughter and humor is 
indicative of this (Glenn, 2003). 
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Excerpt 3 highlights a particularly vivid example of collaboration-in¬ 
process. It is through the spontaneous contributions of students, teachers, and 
administrators that this story of collaboration is told. In the telling, we see 
participants reveal personal aspects of who they are and their relationships with 
one another. John’s laughter and apparent joking stance indicate an informality 
and closeness in his relationship with his students. He has not only negotiated an 
aspect of his curriculum with them as has been reported, he appears to be 
comfortable in this role. Apparent intimacy can also be seen in students’ 
relationships with one another as they laugh together (Lines 6, 11, 18, 24) expand 
on one another’s ideas (e.g., Student 3’s addition in Lines 19-20 to the story 
initiated by Student 6), and agree with one another (Line 35). Aspects of 
participants’ intellectual tastes are additionally made known: not only do students 
reveal their feelings about a particular novel, so does Chris, a college 
administrator who has just met the students for the first time. We also see Chris as 
an empathic listener when he expresses how challenging this book must be for the 
students to read (Lines 14-15 and Line 23), and utters a supportive backchannel 
(Line 21). Chris’ personal investment in the discussion is further found in his 
exclamation of surprise in response to the news that the group stopped reading an 
assigned text. He is growing his understanding of students’ experiences in the 
program. All of these contributions indicate participants’ commitment to this 
communal conversation. As students, teachers, and administrators talk together 
with the joint aim of reflecting on the LC experience, the formation of a new 
group dynamic can be seen. The establishment of this growing community, in 
turn, affects the quality of the reflective conversation, encouraging participants to 
share more of who they are. 

The conversation analysis examples of Excerpts 1, 2 and 3 illustrate two 
central findings of our study: collaboration-in-content and collaboration-in- 
process. Students, as part of communal conversations with other LC stakeholders, 
were found to be affirming a foundational theoretical principle of their program: 
collaboration is important to them for learning. In the content of their utterances, 
students were articulating the value of their collaborative learning experiences, 
and demonstrating a growing expertise at making clear distinctions between these 
experiences and the kinds of interactions occurring in classrooms outside of their 
learning community. While students also had positive things to say about 
collaboration in their individually produced written work, responses in this 
context were not as generative. Students seemed more concerned with saying the 
“right” thing to an academic audience than in probing more complex aspects of 
their LC participation. 

When we initiated our two-semester ESL LC model, one concern expressed 
by college administrators was that its inter-leveled structure might have negative 
effects, with weaker students having the potential to hold back stronger students. 
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In the communal conversations, however, this issue did not come up. Students 
talked instead about the benefits of working with their peers, using language that 
drew attention to their identity as a group (the term “family”; the pronoun “we”) 
in coming together to overcome challenges. Collaboration-in-content examples 
included the help they provided one another in completing “harder... assignments” 
(even calling each other “really late,” or at “4:00 am, 5:00 am,” to do this), and 
the moral support they gave to one another, as when it was time to do a 
presentation in front of class in their Speech course and there could be fear, but 
“everyone actually.. .help” and say “okay go up.” Rather than portraying 
classmates through the lens of individuals’ language “levels,” students were found 
to talk about one another as whole persons, highlighting one another’s strengths 
and recognizing that there were important roles to play in their community other 
than academic ones. As one student noted about the high achiever in their link: 
“she’s the smarter one, she’s our brain. I’m the heart one.” The programmatic 
suggestions we heard from students that addressed collaboration focused on 
providing additional collaborative opportunities through a further extended LC 
experience. One student asked for linked cohorts to be together for “two years, 
three years,” connecting students with the same majors together. 

Our finding that these communal conversation data were richer than data 
from students’ individual writings, while notable, was not surprising. Using 
Malnarich, Pettitt, and Mino’s (2014) Peer-to-Peer Protocol as a guide, students 
were asked to use their writings as a tool for thought within the discussions, and 
conversation facilitators were actively promoting students’ deeper explorations of 
the principles they had written about. What did stand out to us, however, was just 
how rich these data were—how much the organic format of the tum-by-turn 
context appeared to lend itself to the production of students’ more honest and 
thoughtful explorations. This fuller picture of students’ LC experiences emerged 
with the input of others who, through dialogue, were demonstrating their interest 
in learning from them—peers, and also members of students’ broader campus 
community. In these collaboration-in-process interactions, we saw the language 
from which conversations are made: supportive utterances, agreement tokens, 
laughter, interjections, elaborations, jokes, and self- and other- corrections. 
Participants demonstrated their high involvement in one another’s contributions in 
their echoing of words and ideas, utterance latching, overlap, and co-construction 
of thoughts across speaking turns (Tannen, 2005). These communal 
conversations, evidently a natural extension of students’ instructional 
conversations (Tharp & Gallimore, 1988) in their LC classrooms, point to the full 
range of what a dynamic program assessment can reveal about the learning 
community experience. 


Discussion 
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According to Vygotskyan scholars James Lantolf and Matthew Poehner 
(2014), “the true test of their [Vygotsky and colleagues’] theory... was determined 
by the power of the theory to make a difference in the practical behavior of a 
community” (p. 27). In embarking on this study, we wanted to test our theory of 
the social origins of teaching and learning by hearing from our students about 
whether or not “the practical behavior” of their learning community reflected this 
underlying view. Taking a step back to look more closely at the core principles of 
our ESL Learning Communities with our students, we knew, would give us the 
opportunity to think more about, and possibly rethink, aspects of these principles, 
and their applications in practice. 

Community-in-content and community-in-process data provide evidence 
that the educational theory grounding the design of our ESL LCs is serving our 
students well. In these data, students demonstrated their support of our 
collaboration principle in the examples they shared of working closely and 
productively with their professors, tutors, and one another. They also showed their 
support for this principle by being expert collaborators themselves, revealing a 
history of productive dialogue in their communal conversations. Data from these 
lively conversations made it clear to us that students had become comfortable 
collaborating through social interaction over time as part of their two-semester 
learning community experience, and that these interactions were providing them 
with fruitful opportunities for development. While, as LC practitioners, we knew 
that collaboration is important for learning, as a result of this dynamic program 
assessment, our commitment to a view of learning as a social, deeply human 
process, has become stronger. We are more aware of how complex, nuanced, and 
wide-ranging the effects of LC collaboration can be. This awareness grew from 
our own participation in an assessment experience that allowed us to collaborate 
with our students. The valuable feedback we received came about through shifts 
in traditional classroom roles. In our communal conversations, students were 
experts, and teachers/assessors became listeners and learners. This shift is also at 
the heart of the LC movement. Teachers in learning community classrooms are 
not understood to be the sole authority, but rather individuals who share 
responsibility for learning with their students, working to create spaces in which 
learners can “discover, construct, and reconstruct their understandings of the 
world” (Malnarich, G., with others, 2003, p. 38). 

Evidence was provided in our communal conversations that teachers and 
administrators were not only sharing the speaking floor with students, but also 
listening to what students had to say. A teacher was told by his students that they 
didn’t like a course text, and he changed it. A college administrator, making a 
personal connection to this text, was impressed that the change had occurred. A 
program director signaled to students that she was hearing their contributions, and 


19 



Learning Communities Research and Practice, Vol. 4 [2016], Iss. 2, Art. 2 


encouraged them to speak more. From their individual perspectives, participants 
in the group were found to be blurring traditional lines between teacher, 
administrator, and student. In this way, our dynamic program assessment can be 
seen as an extension of the LC mission. Bringing various campus representatives 
together for a collaborative discussion that informs the group as a whole reflects 
the view that the roots of quality higher education are found in community. 

Conclusion 

This study challenges the mainstream view of academic assessment as a 
static and discrete practice that focuses on examining lag measures—historical 
measures that are easily captured but difficult to influence directly (McChesney et 
al., 2012)—such as students’ grades, retention and graduation rates, and progress 
through curricular sequences. After analyzing these types of measures with an 
interest in assessing the quality of ESL Learning Communities at our college, we 
were still left with several key questions: Which components of the program 
should be reinforced? Which should be modified? How can we improve our 
students’ experiences? While we take pride in these metrics showing students in 
two-semester ESL LCs progressing at faster rates through their developmental 
English coursework, we are also aware that these results do not offer much 
meaningful feedback on whether the principles behind the program actually 
maximized students’ opportunities for learning in their classrooms. This is 
particularly true since our program principles reflect a view of learning that is 
understood not to be adequately measured by an individual’s solitary performance 
on a test or task, but, more importantly, by what learners can do with others’ 
assistance: an understanding of learning as a collaborative, fluid, and ongoing 
process (Vygotsky, 1978). 

Malnarich, Pettitt, and Mino’s (2014) Peer-to-Peer Reflection Protocol 
allowed us to examine key curricular questions that reflected the sociocultural 
principles central to classroom practices in our learning communities. The PRP 
also gave us the opportunity to explore a view of assessment that speaks to us as 
learning community practitioners: the unification of instruction and assessment 
into the same whole process through dynamic assessment (Lantolf & Poehner, 
2014). In adapting the PRP as a tool for our learning community program 
assessment, we were interested both in helping our students arrive at new 
understandings about their experiences in our program as part of a group 
dialogue, and in obtaining valuable feedback from students in our ongoing efforts 
to tailor our programmatic work to their needs. 

For this assessment activity, we invited a college administrator from our 
Institutional Research office to participate in the discussion with students, 
teachers, and program directors. Learning communities, by providing 
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opportunities for interdisciplinary collaboration, break down the barriers erected 
by higher education organizational structures which assign instructors to separate 
academic departments. Yet, foundational to the success of LC programs is not 
only collaboration among faculty, but also collaboration between faculty and 
campus administrators. We wished to explore an assessment methodology 
designed to bring these various LC representatives together. As assessment is too 
often considered a task produced for an outside audience as opposed to an activity 
informing pedagogical practice and program development, we hoped our research 
would speak to our LC work on the ground, and also inform our larger campus 
community about this work. 

By including these various individuals in an intentionally designed, 
reflective conversation about the LC experience, program assessment became a 
dialectical cooperative process that was integrated into pedagogical practice rather 
than imposed on it. Data from our communal conversations showed the IR 
administrator being folded into learning communities by participating students 
and teachers, changing his potential ‘outsider’ status to one of ‘insider’ as he 
searched for meaningful qualitative feedback about an LC program. We see the 
implementation of this kind of inclusive assessment framework as having the 
potential to affect college-wide policymaking in positive ways. Because Chris 
participated in the PRP process, he has become more deeply aware of the value of 
incorporating multiple voices into campus-wide program assessments—not just 
hearing these voices, but interacting with them. He sees the bridging of theory and 
practice that occurred through such interactions in this study as not only having 
the potential to drive pedagogical and administrative change in our learning 
community classrooms, but also to broaden the ways in which we define and 
practice the activity of assessment itself. 

Our dynamic program assessment framework was designed to springboard a 
conversation that would not only be enlightening to us, but also educational for 
our students. In our adaptation of Malnarich, Pettit, and Mino’s (2014) PRP, we 
therefore sought to mirror our typical LC classroom practices as closely as 
possible. Review is especially important to us as ESL instructors, so one aspect of 
the assessment activity involved asking students to read aloud the program 
principles they had written about. Additionally, as we usually ask students to 
work together in small groups before coming to a teacher-fronted discussion, 
students shared a piece of their writing in pairs before joining the whole-group 
listening round. We also carved out space for ourselves in the discussion to point 
out possible connections we saw between students’ contributions, and curiosities 
we had as teachers and administrators, to encourage students to dig more deeply 
into aspects of their LC experiences. The quality of the feedback we received 
through all of these activities—an extension of students’ regular LC activities— 
has given us the opportunity to initiate important conversations with colleagues 
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about the quality of our work as LC professionals. Reflective of the assessment 
protocol we followed, we see how critical it is that we design our lesson plans 
with the aim of opening up classroom conversations in intentionally designed 
ways. We will be bringing the topic of group reflection to a faculty development 
session, and are considering a new principle for our ESL LCs addressing the 
relationship between reflection and learning. Findings from this study have also 
informed our understanding of students’ needs once they exit the ESL Learning 
Community setting. Based on students’ feedback, we are working with other 
members of Kingsborough’s Learning Community Program team to give as many 
students graduating from ESL LCs as possible the opportunity to enroll in new 
LCs for continuing students based on their majors. 

Another aspect of our dynamic program assessment framework was the use 
of a data collection and analysis process that captured the richness of conversation 
as it emerged. By audio-recording the communal conversations, we could listen to 
them, and listen again. What we heard was participants doing the activity of 
collaboration in their conversational details. Students’ individual thinking 
processes were also revealed in their speech. Evidence was provided of how 
students learn, and how communities are built. 

Perhaps above all, the close examination of our communal conversation 
transcripts revealed a finding that we already know as learning community 
practitioners, but discovered that we still have a lot to learn about: “not to be 
overlooked or underestimated, students' sense of community seems to be 
foundational to [their] exploratory talk” (Malnarich, Pettitt, & Mino, 2014, p. 18). 
In their communal conversations, we saw—through product and process—how 
important students’ relationships with one another and their teachers were. This 
finding continued to resonate in the year subsequent to our data collection. While 
they had moved on from their first-year LC program, students were still found to 
display a strong desire to continue their collaborative educational experience in a 
multitude of ways. They became academic mentors to incoming ESL LC students, 
and enrolled in new classes taught by previous LC professors. They visited past 
instructors with questions about present and future college classes, and continued 
to offer academic support to classmates from their ESL LCs. As we continue to 
evolve our learning communities, we will continue to remind ourselves of the 
lasting effects that strong collaborations can have on our students’ futures. 

In sum, we offer three interrelated conclusions from this investigation. First, 
programs as dynamic as learning communities are best assessed dynamically. 
When asked to reflect on their learning experience, students of a program 
designed around the view that knowledge is socially constructed will provide 
more thoughtful assessments in a cooperative, community setting than in 
individual, isolated writing exercises. Second, the program assessment practices 
we engaged in contribute to the potential broadening of existing definitions of 
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assessment. Based on the results of this study, we argue for a program assessment 
framework that goes beyond being “the systematic collection, review and use of 
information about educational programs” (Palomba & Banta, 1999, p. 3), and 
becomes an integral part of the teaching and learning process itself. Third, data 
from this research highlight the importance of broad participation in the 
assessment of LC programs. Participants in our dynamic program assessment 
brought their sociocultural and positional identities into the conversation as well 
as their different perspectives and questions. Each communal conversation 
member contributed significantly to the co-constructing of meaning as ideas were 
built across speaking turns. In moving from the linear data collection-analysis- 
strategic pedagogical change model of traditional academic assessment to a 
dialectical model of dynamic program assessment, conventional lines between 
teacher/student/assessor are blurred. To apply Tonnies’ (2001) sociological 
terminology, our dynamic program assessment allowed the group to evolve from 
students participating in a learning community and the professionals assessing 
their experience (gesellschaft - society) to a true community of learners 
(gemeinschaft - community). 

Based on our implementation of a dynamic learning community program 
assessment using Malnarich, Pettitt, and Mino’s (2014) Peer-to-Peer Reflection 
Protocol, we encourage practitioners in other LC programs to take up the authors’ 
call to draw upon this tool with the hope that, as a field, we leam more about 
students’ collaborative learning experiences. Our adaptation of this Protocol gave 
us the opportunity to reconnect with the theory of collaborative learning at the 
heart of the learning community mission through the activity of collaboration 
itself. As we explored sociocultural theory in practice, we educated one another 
about what learning communities are, and what they can do—students, teachers, 
and assessors—together. 
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Appendix A: Peer-to-Peer Reflection Protocol 

This peer-to-peer reflection protocol is a companion tool to the Survey of 
Studen ts ’ Experiences of Learning in Learning Communities. Its aim is to provide 
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LC Programs with a method which invites students—in the company of peers—to 
further explore their experiences and perceptions of integrative learning as well as 
collaborative learning. 

Designed to be used after students do the online survey, the protocol builds 
on individuals’ anonymous survey responses. It has two distinct and related parts. 
The first is a written reflection exercise based on a section from the online survey 
which focuses on integrative learning. The second uses what students’ write as a 
starting-point for a facilitated, exploratory conversation where the intent is to 
deepen students’ collective understanding of their learning community 
experience. 

Peer-to-Peer Reflection Protocol (PRP) for Exploring Students’ LC 
Learning Experiences 

Part I - Post-Survey Written Reflection Exercise 

1. Once the online survey has been administered, distribute the handout 
in the box below which asks each student to provide examples or situations 
which support their rating of two questions from the survey most related to 
integrative learning. This post-survey written reflection exercise can be done 
in the same class or in a time-frame which suits the LC Program. You will 
want to add additional spacing after each prompt; the handout should be 
around one and a half to two pages in length. 


Directions: For each of the survey questions below, provide one or two 
examples or occurrences in your learning community that came to mind 
as you were responding to the question. 

In my learning community, I: 

a. Work on connecting or integrating ideas, strategies, or skills from 
classes (or disciplines) included in this learning community 

b. Reflect on how these connections lead to new insights or understanding 


2. After all students have had a chance to write responses, facilitate a 
discussion with students based on their responses and insights (see page 2). 
This can be done in the same class or in a time-frame which suits the LC 
program. We recommend that facilitation be done by a member of the 
teaching team or someone knowledgeable about the LC class. 

3. You may want to note down or record highlights of the discussion for 
future in-class use and/or professional development conversations. 


http://washingtoncenter.evergreen.edu/lcrpjournal/vol4/iss2/2 
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Part II - Facilitated Discussion 

The purpose of this part of the protocol is to provide an opportunity for 
students to develop a collective understanding of their learning community 
experience by considering both the individual and collaborative nature of learning 
in LC classrooms. 

1. Introducing the process 

The facilitator reviews the role of the facilitator, i.e. to make sure the 
conversation stays focused on each step in the reflective discussion: 1) listening 
round; 2) making connections; 3) wondering; and 4) uncovering complexity. 

2. Discussion agenda 

Listening round. One by one, without intermption, students share examples 
or situations from their written responses in a round robin fashion. 

Making connections: Once everyone has shared their responses, the 
facilitator asks students to point out any connections or common themes given the 
range of experiences and situations. 

Wondering: Students are invited to raise questions or name anything they 
are curious about in relation to the kind of learning their fellow students are 
describing. 

Uncovering complexity: The facilitator invites everyone present to share any 
thoughts they have about “what lies beneath the surface” of their experience. 

3. Appreciating students’ contributions to the LC Campus Program 

The facilitator thanks students for sharing their insights with classmates and 
their contributions to improving the LC program on campus. 
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